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Abstract 

It is shown for classical and quantum ensembles that there is a 
unique quantity which has the properties of a "volume" . This quantity 
is a function of the ensemble entropy, and hence provides a geometric 
interpretation for the latter. It further provides a simple geometric 
picture for deriving and unifying a number of results in classical and 
quantum information theory, and for discussing entropic uncertainty 
relations. 
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Classical and quantum ensembles make natural appearances in many dif- 
ferent contexts. These include, for example, equilibrium ensembles in statis- 
tical mechanics; signal-state ensembles in communication theory; dynamical 
ensembles corresponding to Brownian or chaotic diffusion; and ensembles 
corresponding to the classical limits of various quantum states. 

In each of the above contexts it is often fruitful to employ the notion 
of a "volume" associated with the ensemble. For example, derivations in 
statistical mechanics often involve counting "microstates" in a volume of 
small thickness containing a constant-energy surface Shannon's theorem 
for information transfer, in the case of signals subject to quadratic energy 
and noise constraints, can be proved by considering the ratios of spherical 
volumes in high-dimensional spaces |Q] . In Ornstein-Uhlenbeck diffusion the 
evolution of a Gaussian ensemble is usefully depicted by a "distribution 
ellipsoid" (with principal axis lengths corresponding to root-mean-square 
variances) |^ , where the ellipsoid volume is a clear measure of the "spread" 
of the ensemble. Finally, minimum phase-space volumes on the order of /i" 
play a fundamental role in the classical limit of quantum mechanics . 

The above examples raise the question of whether there is in fact some 
general measure of "volume" for classical and quantum ensembles, which 
may be usefully employed in all of the above contexts but which is not re- 
stricted in application or interpretation to various special cases. Thus, for 
example, the measure AxAp for an ensemble of systems with a 2-dimensional 
phase space is not suitable, because (i) it is not invariant under canoni- 
cal/unitary transformations; (ii) it has no natural generalisation to finite- 
dimensional quantum systems; and (iii) while it leads to useful geometric 
information bounds |^, these are only exact in particular cases. 

It will be seen here that there is indeed a natural measure of "ensemble 
volume". This measure is a function of the Gibb's entropy for classical 
ensembles and of the von Neumann entropy for quantum ensembles, and 
hence provides a unified geometric interpretation for these quantities. 

Before writing down this volume measure, it is useful to consider four 
simple properties which uniquely define it (up to a normalisation constant). 
The first is an invariance property, while the remaining three are geometric 
in nature. It is convenient to state these properties in a form independent 
of whether the ensemble is classical or quantum, as a common geometric 
viewpoint is desirable. Thus, T will be allowed to denote either a classical 
phase space or a quantum Hilbert space; p either a probability distribution 
on phase space or a density operator on Hilbert space; and Trr[-] either 
integration over a classical phase space or the trace over a quantum Hilbert 
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space. Moreover, will denote the phase space or Hilbert space corre- 
sponding to an ensemble of composite systems with subsystem spaces Fi 
and i.e., denotes Fi x r2 for classical ensembles and Fi (g) F2 for 
quantum ensembles. 

Consider now a volume measure V{p) which satisfies the following prop- 
erties: 

(i) Invariance Property: V{p) is invariant under all canonical trans- 
formations (these are represented by unitary transformations for quantum 
ensembles). Note that this property ensures that the volume is a function 
of the ensemble alone, independently of a particular co-ordinatisation or 
measurement basis. 

(ii) Cartesian Property: If p describes two uncorrelated ensembles pi and 
P2 on Fi and F2 respectively, then 



This property is exactly analogous to the geometric property that area equals 
length times breadth, and is illustrated in Fig. 1. Note that p is the product 
P1P2 for classical ensembles, and the tensor product pi®p2 for quantum 
ensembles. 

(in) Projection Property: If p describes an ensemble of composite systems 
on F12 then 



where pi, p2 denote the "projections" of p onto Fi, F2 respectively (i.e., pi = 
Trr2 [p] , P2 = Trr^ [p] ) . This property is exactly analogous to the geometric 
property that a volume is less than or equal to the product of the lengths 
obtained by its projection onto orthogonal axes, and is illustrated in Fig. 2. 
Note that pi , p2 denote the marginal distributions corresponding to Fi , F2 
for classical ensembles, and the reduced density operators corresponding to 
Fi, F2 for quantum ensembles. 

(iv) Uniformity Property: If p' and p" denote two non-overlapping en- 
sembles (i.e., Trr[p'p"] = 0)) with equal volumes V{p') = V{p") = V , then 
an arbitrary mixture p of p' and p" has volume no greater than 2V , where 
the latter corresponds to the volume of an equally- weighted mixture, i.e., 



V{p) = V{pi)V{p2). 



(1) 



V{p) < V{pi)V{p2) 



(2) 



V{p) < V{p'/2 + p"/2) = 2V. 



(3) 



Thus uniform mixtures maximise ensemble volume. 
One has the following result: 
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Theorem: Any (continuous) measure of volume satisfying properties (i)- 
(iv) above has the form 

V{p) = K{T)e'(P\ (4) 
where S{p) denotes the ensemble entropy 

S{p) = -Trriplnp], (5) 

and K{T) is a constant which may depend on F, and satisfies 

K{ru) = i^(ri)i^(r2). (6) 

The proof will be given elsewhere for reasons of space, and primarily 
relies on applying properties (i)-(iv) to an arbitrarily large number of inde- 
pendent copies of a given ensemble p To indicate its plausiblity here, 
consider a quantum ensemble with eigenvalue distribution {Xj}. Properties 
(i), (ii) and (iv) are certainly satisfied by the "Renyi" volumes (J^ji^j)'^)^ 
with (1 — a)P = 1, together with a term K{T) satisfying Eq. (|6|). A result 
of Renyi implies then that the projection property (iii) is satisfied only 
in the limit a ^ 1, yielding Eqs. (^) and (^). 

The universal geometric interpretation of the volume measure in Eq. (^) 
contrasts with ensemble entropy, for which the only context-independent 
interpretation to date appears to be as a somewhat vague measure of "un- 
certainty" or "randomness" |0, |8|, |^ , which has a limited and purely heuristic 
usefulness. Indeed, it is perhaps conceptually more useful to define ensem- 
ble entropy as the logarithm of the ensemble volume V{p), thus providing 
a geometric picture for many of its properties. Applications of the volume 
measure in Eq. (^) to various contexts will now be discussed. 

First, in the statistical mechanics context, the Gibbs relation Sth = 
kS{p) between thermodynamic entropy and ensemble entropy for equilib- 
rium ensembles can be rewritten via Eqs. ^ and (^) as 

Sth = kHV{p)/K{T)]. (7) 

Thus, the thermodyamic entropy is (up to an additive constant) proportional 
to the logarithm of the ensemble volume. Note further from Eq. (^ and the 
third law of thermodynamics (that thermodynamic entropy vanishes at ab- 
solute zero), that KiV) should correspond to a minimum "zero-temperature" 
ensemble volume. For quantum ensembles one has from Eqs. (§) and (^) 
that V{p) = K{T) for pure states, i.e., the quantum zero-temperature vol- 
ume is just that of a pure state on F. However, classical ensembles violate 
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the third law ||^] and K{T) remains arbitrary in this case (but see Eq. ( p^q ) 
below) . 

The geometric expression is very similar to the original Boltzmann 
relation Sth = A;lnl^ , where W is the number of distinct microstates or 
"elementary complexions" consistent with the thermodynamic description. 
Indeed, from the above discussion it follows that Eq. (0) provides a precise 
geometric interpretation of the Boltzmann relation for equilibrium ensem- 
bles: thermodynamic entropy is proportional to the logarithm of the number 
of non-overlapping zero-temperature (or "microstate" ) volumes contained 
within the total volume of the ensemble. 

Second, in the communication context, consider a communication chan- 
nel where signal states pi, p2, ... are transmitted with prior probabilities 



Pi, P2, ■ ■ ■ respectively [|10|. The ensemble of signal states itself corresponds 
to the mixture 

i 

As a warm-up exercise, suppose that the signal states all have volumes 
greater than some minimum volume Vq (e.g., due to noise in the channel). 
From the uniformity property (iv) it follows that the maximum possible 
number of non- overlapping signal volumes available for a given signal en- 
semble p is bounded by V{p)/Vq. Hence the maximum amount of error-free 
data, /i, which can be gained by a single measurement at the receiver (mea- 
sured in terms of the number of binary digits required to represent the data) , 
is bounded by 

h < log2{V{p)/Vo). (9) 

For example, if the channel is quantum one may always take Vq = K{T) 
(i.e., the volume of a pure state). Eqs. (Q) and (^) then immediately yield 
the single-measurement quantum information bound 

I? < S{p) log2 e, (10) 

which may be recognised as a special case of Holevo's bound in quantum 
communication theory |jlT[ (see also below). 

More generally, one may seek to improve information transfer by coding 
data into blocks of signal states, of some length L, and restricting trans- 
mission to particular blocks of signals These may be referred to as 
"block-signals", to distinguish them from the individual signals {pi}- The 
ensemble of block signals will be denoted by p^^\ Under the constraint 
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that state pi still appears with relative frequency pi per individual signal 
transmission, one has 

V{p^'^^)<V{-p,)...V{-pL)<[V{pt, (11) 

where is the average Z-th component of the transmitted block-signals; the 
first inequality follows from the projection property (iii); and the second 
inequality from the concavity property 

Y.L-'S{-pd<S{Y.L-'-pd = S{p) (12) 

of entropy and Eqs. (|), (||) and (§). 

Further, for L sufficiently large, the strong law of large numbers guaran- 
tees that most block-signals are "typical" in the sense of Shannon [^, i.e., 
the number of times that a signal state pi appears in a given block-signal is 
approximately PiL, except for a set of block-signals with a total probability 
of occurrence approaching zero as L is increased (and hence which may be 
ignored). It then follows immediately from the Cartesian property (||) that 
the volume of a typical block signal a is approximately 

Va = \{[vip^)r'' (13) 

i 

(and approaches it arbitrarily closely in the limit of large L). 

Exactly as per the derivation of Eq. (^), the amount of error-free data 
/ which can be gained per individual transmitted signal, by measurements 
on block-signals, is geometrically bounded in the limit of arbitrarily large 
block size by 

/<L-ilog2[F(p(^))/K.]. (14) 
From Eqs. (§), (|ll|) and ( |l3|) this yields the general upper bound 

I<[S{p)-Y,nS{p,)]\og^e. (15) 

i 

For classical ensembles, Eq. (|l5|) may be recognised as the Shannon hound 
for discrete memoryless channels [0], while for quantum ensembles it may be 



recognised as the Holevo hound for such channels |11, |l^, |l^. The unified 
derivation of these bounds from simple volume considerations is a central 
result of this paper. It is particularly valuable in the quantum case, as 
existing derivations of the Holevo bound are mathematically rather technical 
[U], P, PI. 
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It has recently been shown |15| that the Holevo bound (15) is in fact 
tight, in the sense of being achievable arbitrarily closely for sufficiently large 
block sizes by choosing a suitable set of typical block-signals for transmis- 
sion and making appropriate measurements on these block-signals at the 
receiver. Geometrically, this result corresponds to being able to choose up 
to [^(/o)]^/V^ non-overlapping block-signal volumes, in the limit of arbitrar- 
ily large L, and it would be valuable if a geometrically-based proof of this 
could be given. Since the total number of typical signal-blocks may be es- 
timated as exp(— L Pi Inpj) |8|, tightness of bound Eq. ( [l5| ) immediately 
implies the Lanford-Robinson inequality ||l( 



i i 

Next, in the diffusion context, consider a Gaussian classical ensemble 
with root-mean-square variances Axi, . . ., Ax„ , Api, . . ., Apn relative to 
the principal axes (these variances and axes will in general vary in time for 
Ornstein-Uhlenbeck diffusion processes The ensemble volume may then 
be calculated from Eq. (Q) as 

n 

V{p) = K{T) Yl 2TTeAxjApj, (17) 
i=i 

which is directly proportional to the volume of the corresponding "distribu- 
tion ellipsoid" (with equality for the choice K{T) = n!(2e)~"). This volume 
is moreover invariant under all canonical transformations. Note that triv- 
ially from Eq. (^) irreversible dynamical processes are characterised by a 
strictly increasing ensemble volume. 

Consider finally a classical ensemble pc which is the "classical limit" 
of some quantum ensemble pq, i.e., the physical properties of pc approxi- 
mate those of Pq. For the case of a spinless particle on a 2n-dimensional 
phase space one can obtain a relationship between the constants K{Tc) 
and K(Tq) in Eq. by requiring that the ensemble volumes V{pc) and 
V{pq) are approximately equal for such ensembles. Since these constants 
are independent of the dynamics of the ensemble it suffices to choose an 
equilibrium ensemble of isotropic oscillators. Equating V{pc) and V{pq) in 
the high-temperature limit then yields 

K{Tq) = h^K{Tc), (18) 

for the volume of a pure state, where h is Planck's constant. 
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Eq. ( p!8[ ) can be used to derive semi-classical uncertainty relations from 
geometric considerations. For two corresponding ensembles pQ and pc as 
above the position and momentum entropies S{px) and S{pp) will be ap- 
proximately equivalent for either ensemble, and further 

eMS{px)) exp(S(/5p)) > exp(5(pc) (19) 

holds for the classical ensemble from the projection property (ii) applied to 
projections onto the position and momentum axes rather than spaces Ti 
and r2 0. Eqs. (|), (0) and (0) yield 



S{px) + S{pp)^n\nh. (20) 

This semi-classical entropic uncertainty relation, holding for quantum en- 
sembles which have classical limits, arises directly from the existence of a 



minimum volume for quantum ensembles. Following the method of |18|, the 
corresponding semi-classical Heisenberg uncertainty relation 

AxAp ~ n/e (21) 

follows for the n = 1 case. Eqs. (|0D and (|l]) are close to the exact results for 
general quantum ensembles ]l^ (where, e.g., e is replaced by 2 in Eq. (pT|)). 

In conclusion, an essentially unique measure of volume for classical and 
quantum ensembles has been found, related to ensemble entropy, which pro- 
vides a geometric tool for any context in which ensembles appear. This mea- 
sure is universal in the sense that it may be defined by theory-independent 
concepts of invariance, uncorrelated ensembles, projection, and non-overlapping 
ensembles (properties (i)-(iv)). Applications include a precise geometric in- 
terpretation of the Boltzmann relation; a unified derivation of results in clas- 
sical and quantum information theory based on simple geometric properties; 
an invariant generalisation of "ellipsoid volume" for diffusion problems; and 
a geometric derivation of semi-classical uncertainty relations. 

This work was supported by the Alexander von Humboldt Foundation. 
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FIGURE CAPTIONS 



FIG. 1. Two uncorrclated ensembles pi and p2 on spaces Fi and F2 re- 
spectively (shown here compressed to 1-dimensional axes), have respective 
volumes V{pi) and V{p2) as indicated by the darkened axis regions. The 
Cartesian property Eq. (1) states that the corresponding joint ensemble p 
has a "rectangular" volume V{p) = V{pi)V{p2), i.e., V{p) corresponds to 
the Cartesian product of volumes V^(pi) and V{p2)- 

FIG. 2. An ensemble p on the product space of Fi and F2 has a volume 
V{p) indicated by the solid closed curve. The corresponding projected (or 
"reduced") ensembles pi and p2 on Fi and F2 respectively have projected 
volumes V{p\) and V^(p2), indicated by the darkened axis regions. The pro- 
jection property Eq. (2) states that V{p) can be no greater than the volume 
of the rectangular region formed by the dashed lines, i.e., than the product 
of the projected volumes. 
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